Maximum Entropy and Bayesian Data Analysis: Entropic Priors

摘要

The problem of assigning probability distributions which objectively reflectthe prior information available about experiments is one of the major stumblingblocks in the use of Bayesian methods of data analysis. In this paper themethod of Maximum (relative) Entropy (ME) is used to translate the informationcontained in the known form of the likelihood into a prior distribution forBayesian inference. The argument is inspired and guided by intuition gainedfrom the successful use of ME methods in statistical mechanics. For experimentsthat cannot be repeated the resulting "entropic prior" is formally identicalwith the Einstein fluctuation formula. For repeatable experiments, however, theexpected value of the entropy of the likelihood turns out to be relevantinformation that must be included in the analysis. The important case of aGaussian likelihood is treated in detail.